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ABSTRACT 

Teacher behavior is not likely to be "valid" in 
general^ but rather only under specified conditions^ such as the 
nature of the learning objective which is being pursued^ the personal 
and demographic characteristics of the pupil being taught^ the length 
of tiae during which learning takes place^ and probably the context 
of other classroom behaviors in which the given behavior is 
experienced. The need for specifying what^ for whoa^ how long^ and 
what else^ in teras of the application of validity specif ications, 
are recognized in the Georgia project. The type of achievement 
measures selected and data obtained to meet this need call for an 
analysis which recognizes the influence of multiple measures 
simultaneously, in the Georgia project, data analysis will therefore 
be by iaultiple regre^ssion and, where adequate degrees of freedom are 
available, by multivariate analysis. The project will attempt to 
minimize the problem of inconsistency by replicating results over 
several years, sharpening hypotheses, and refining measurement 
procedures. Although past work in measuring teacher behavior and 
relating it to measures of pupil growth has not produced extensive 
agreement on findings, the improvements in methodology which are 
needed have become clearer. Hore rapid progress may now be possible 
where past progress has been slow and uncertain, and the Georgia 
project will be in a position to profit from these advances. (PB) 
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Although the major focus of this paper is validation of competencies, a 
word about the origin of the competencies studied in the Georgia project seems 
in order. Competency lists which had been developed in a number of states were 
reviewed by a committee of Carroll County teachers and competencies were selected 
which they felt should be included for certification, and they were rank-ordered 
in relative importance. i\hen reviewed by a larger group of teachers in Carroll 
County, agreement with the competency list and tha rank-order was high* 

After the competencies had been identified, existing observation instruments 
were reviewed as sources of behavioral measures. Five observation instruments 
with relatively extensive research histories were selected for use, from which 
behavioral neasures of numbers of the competencies could be abstracted, although 
it was not possible to represent all competencies* Data collection with those 
instruments is currently underway in classrooms. 

A central element in the rationale for both competency selection and valida- 
tion has been the assumption that a competency-based certification system could 
ultimately be expected to stand court-test, so that the data on which certification 
was based needed to be as objective as possible. This meant that "counting measures" 
should be strongly preferred, if not obligatory, and that establishment of the 
relations bet;%een these measures of teacher behavior and measures of change in 
pupils — validation — \/as centrally important. Recent research in teacher effective- 
ness indicates that there are a number of challenges which this validation effort 
must meet if it is likcl}" to be meaningful* 



^Presented in a s>Tnposium titled "Mandated Competency-Based Teacher Certification 
and the F'ublic Inteiest*', American Educational IU.scarch Association, Washington, D.C., 
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Validation of Competencies 

Validity Specification 

An over-all orientation which is developed in more detail elsewhere (Medley, 
Soar and Soar, 1975) is that a teacher behavior is not likely to be "valid" in 
general, but rather to be valid only under specified conditions — such as the 
nature of the learning objective which is being pursued, the personal and demo- 
graphic characteristics of the pupil being taught, the length of time during 
which the learning takes place, and probably the context of other classroom 
behaviors in which the given behovior is experienced. 

The Nature of the Ob jective - It does not seem probable that the same teacher 
behavior is best for pupil growth in all objectives. That is, the teacher behavior 
which facilitates complex cognitive growth such as? abstracting, generalizing, 
inferring, or creative endeavors is probably different from that which facilitates 
memorizing facts such as the multiplication table or the day*s list of fifteen 
spelling words. Several sets of data support this conclusion in varying degrees 
(Solomon, Bezdek and Rosenberg, 1^63; Soar, 1968; Soar and Soar, 1972; Soar and 
Soar, 1975). If this is true, or unless we are sure it is not true, it seems 
important to collect achievement measures which represent varying degrees of 
cognitive complexity and measures of non-cognitive change as well, against the 
possibility that desirable chaiigec for one may not parallel those for the others. 
It would follow, then, that observational data should be recorded separately for 
the teaching of different objectives'. 

Pupil Differejues in Response to Classroom Behavior - It does not seem prob- 
able that the same teacher behavior is best for all pupil groups. Recent evidence 
suggests that a teacher behavior which is best for high socioeconomic status 
pupils is not always besc for low socioeconomic status pupils. At least three 
recent sets of data agree in indicating that the emotional climate of the class- 



room which is associated with greatest achievement growth is different for pupils 
who differ in socioeconoinic status (Brophy and Evertson, 1974; Soar and Soar, 
1973). Scattered findings suggest that other characteristics of pupils may also 
be related to the kind of classroom expeiience which is most functional for them. 

The Time Period for Pupil Growth - Although relatively short time periods, 
ranging from a few minutts to a few weeks, have been advocated as a basis for 
validating competencies, generalizing from short to longer time periods seems 
risky without empirical evidence. It seems probable that it will be easier to 
show the growth of relatively low level cognitive skills in a short time while 
longer time periods would be needed to show growth in more complex cognitive 
skills. Tliere are suggestions tliat the teacher behavior whicli is associated with 
pupil achievement gain during the school year may be different from the teacher 
behavior which is associated with pupil gain the following summer while out of 
school (Soar and Soar, 1973) and there is convincing evidence that practically 
important amounts of pupil growth (as measured by change in standardized achieve- 
ment test scores) occurs over the summer while pupils are out of school (Hayes 
and Grether, 1969; Soar and Soar, 1969). 

The Context of Other Behaviors - It seems intuitively reasonable that the 
impact of any single teacher behavior on pupils will be moderated by the context 
in which it occurs. Some evidence indicates that thio is the case (Soar and Soar, 
1973). For example, the frequency with which the teacher took a central role in 
the activities of the classroom was unrelated to any pupil outcome measure by 
itself, nor was the frequency with which the pupil *s learning task had been 
specified by the teacher. But when the two behaviors were examined simultaneously, 
frequent occurrence of one or the other was associated with increased pupil achieve- 
ment gain, but frequent occurrence of both, or infrequent occurrence of either, was 
associated with decreased pupil gain. Numbers of other such interactive relation- 
ships have been found. 
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Another example of impact being moderated by context occurred when a measure 
of classroom behavior with a near zero correlation with pupil change became 
significantly related to pupil change wlion other measures of classroom behavior 
were held constant statistically. This result presumably parallels the expecta- 
tion in experimental research that the significance of the experimental variable 
can only be tested meaningful 1> when all other variables are held constant. 

Application of Validity Specifications - The need for specifying what, for 
whom, how long, and what else, indicated by the results of past research, are 
recognized in the Georgia project. Tlie staff has selected achievement measures 
which differ in complexity, as well as a limited set of non-cognitive measures; 
they are collecting data on the socioeconomic status of ind5v^idual pupils; they 
are collecting pupil data for the school year as well as for the summer in order 
to relate both to the pupil's classroom experience, and they will examine inter- 
active relationships and the possibility that some teacher behaviors relate to 
pupil outcomes only when others have been held constant. 
Tlie Need for Complex Analysis 

This plan clearly implies the need for an analysis which recognizes the 
influence of multiple measures simultaneously. If interactions between classroom 
behaviors are to be tested, and if interactions of classroom behavior with pupil 
personal and demographic characteristics are to be tested, and if the effects of 
some classroom behaviors are to be examined while holding other classroom behaviors 
constant, the need for a complex analysis is clear. 

Another aspect of the data will need to be dealt with by the analysis. IMs 
is the probability that numbers of classroom behaviors, and possibly some pupil 
characteristics as well, will be related in nonlinear fashion witli pupil outcome 
measures (Solomon, Bezdek and Rosenberg, 1963; Soar, 1968; Coats, 1966; Soar and 
Soar, 1972; Soar and Soar, 1973; Brophy and Kvertson, 1974). This also seems 
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eminently reasonable, once the question is raised. If some of a teacher behavior 
is good, will increasing amounts of it be better, without limit? And yet when a 
product-moment correlation is calculated, this assumption is implicitly made. 
So it seems clear that the data analysis will need to recognize the simultaneous 
effect of considerable numbers of variables, tlie interactions of numbers of 
variables, and also the possibility of nonlinear relationships between variables* 
To complicate things furthei*, the ultimate — nonlinear interactions — have also 
emerged in past woi^k (Anderson, 1970), 

In the Georgia project, analysis of the data by multiple regression is planned, 
which will permit testing multiple variables simultaneously, along with inter- 
actioas, nonlinear relations, or nonlinear interactions (Kerlinger and Pedhazur, 
1973). UTiere adequate degrees of freedom are available, multivariate analyses 
may also be employed (Darlington, Weinberg and Walberg, 1973), 
The Problem of Power 

The limited power of the typical study of teacher effectiveness has not 
generally been recognized until recently. Cohen (1969) argues convincingly that 
in a study which assumes a relation of ,30 between two true measures, a reasonable 
degree of fallibility in measurement procedures would reduce the observed relation 
to ,10. In a stu''^ ' with an N of 15 classrooms, which has not been uncommon, only 
6 out of 100 relations wliich are real w^ould be found to be significant. Much of 
the inconsistency of past results presumably follows from this lack of power. 

The Georgia project will attempt to minimize this problem by replicating 
results over several years, sharpening h>'po theses and refining measurement procedures, 
A Coacluding Comment 

Although past work in measuring teacher behavior and relating it to measures 
of pupil growth has not produced extensive agreement in findings, the improveinents 
in jnethoJology which are needed have become clearer. Progress may now be possible 



where jnust progress has been slow and uncertain, and the Georgia project will 
be in a position to profit from these advances. 

The critical issue for certification in particular, and education in 
general, that empirical validation is a central aspect of the ongoing develop- 
ment, in contrast to programs, however innovative, behavioral, or whatever, in 
which there is no concern for evidence that the effort makes any difference to 

i 

pupili*. That Is an issue whose importance can scarcely be overstated. 
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